Security classification using automated learning (SCALE)

Automating the process of assigning security classifications to unstructured text would facilitate a transition to a data-centric architecture—one that promotes information sharing, in which all data in an organization are electronically labelled. In this document, we report the results of a series of experiments conducted to investigate the effectiveness of using statistical natural language processing and machine learning techniques to automatically assign security classifications to documents. We present guidelines for selecting parameters to maximize the accuracy of a machine learning algorithm’s classification decisions for several well-defined collections of documents. We examine the significance of a document’s topic and the effect of security policy changes on the ability of our system to automate classification; we include design recommendations to address both topic and policy considerations. Our classification techniques prove effective at assessing a document’s sensitivity, achieving accuracies upwards of 80%.